Method and circuit for picture-in-picture superimposition

ABSTRACT

A method for picture-in-picture insertion is described, which is distinguished in particular by the fact that
         the inset pictures are written to a memory device ( 2 ) in a circulating manner under continuously incremented write addresses,   the first and last address of each written-in inset picture is stored,   an overtake signal is formed by comparing the instantaneous address with the previously stored address, said signal indicating whether a previous address has been reached again and, consequently, the corresponding picture content has been overwritten,   by evaluation of the overtake signal, the current or preceding segment is selected for read-out depending on whether or not overtaking took place before the start of the read-out, and   the inset picture stored in the selected segment is read out with continuously incremented read addresses and is inserted into the main picture. A corresponding circuit arrangement is also described.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a method for picture-in-picture insertion, inwhich a sequence of decimated inset pictures is written to a memory withat least two segments and is read out for insertion into a sequence ofmain pictures, to be precise in accordance with the preamble of claim 1,and also a circuit arrangement for picture-in-picture insertion, inparticular for carrying out said method in accordance with the preambleof claim 6.

2. Background Art

Various methods and apparatuses for inserting one or more inset picturesinto a main picture (PIP—Picture In Picture) are known. In this case,the inset pictures stored in the memory are read out synchronously witha main picture. Since the read-out speed is generally higher, in amanner corresponding to the decimation of the inset picture, than thewrite-in speed, a seam can occur in the inset picture on account of thewrite pointer being overtaken by the read pointer, since the insetpicture is then composed of a current part and a preceding part.

In particular in the case where the two parts originate from differentmotion phases, a disturbing effect is produced since moving objectsthrough which the seam passes are displayed in a distorted manner.Moreover, if the frequencies of the inset and main pictures do notcorrespond exactly, the seam drifts, which is perceived as particularlyunpleasant.

EP 0 739 130 A2 discloses, for the purpose of avoiding this problem,storing two inset pictures (or fields in each case) in the memorydevice, so that it is always the case that exactly one picture can beread out while the next picture is written. Although this prevents thewrite pointer from being overtaken by the read pointer, there isnonetheless a significant disadvantage in that the storage capacity mustbe very high, which is associated with considerable costs.

BRIEF SUMMARY OF THE INVENTION

The invention is based on the object, therefore, of providing a methodand also a circuit arrangement for picture-in-picture insertion of thetype mentioned in the introduction by means of which, with relativelylow outlay, an inset picture can be generated without a seam.

This object is achieved by means of a method which is distinguished bythe fact that:

-   -   the inset pictures are written to the memory device in a        circulating manner under continuously incremented write        addresses,    -   the first address of each written-in inset picture is stored,    -   an overtake signal is formed by comparing the instantaneous        address with a previously stored address, said signal indicating        whether a previous address has been reached again and,        consequently, the corresponding picture content has been        overwritten,    -   by evaluation of the overtake signal, the current or preceding        segment is selected for read-out depending on whether or not        overtaking took place before the start of the read-out, and    -   the inset picture stored in the selected segment is read out        with continuously incremented read addresses and is inserted        into the main picture.

The object is furthermore achieved by means of a circuit arrangement forinserting a sequence of decimated inset pictures into a sequence of mainpictures, which has a memory device having at least two segments for theinset pictures, a write controller and a read controller and which isdistinguished in particular by the fact that an overtake signal that canbe fed into a display controller can be generated by the writecontroller, in that a segment buffer is provided, which segment bufferis connected to the read controller and serves to store a first and alast address of an inset picture, and in that a memory segment to beread out by the read controller can be selected by means of the displaycontroller in a manner dependent on the overtake signal.

The solutions according to the invention are based on the insight thatthe write pointer can be prevented from being overtaken by the readpointer through suitable circulating addressing of the memory device ina manner utilizing the speed difference between the pointers. Asignificant advantage of this solution is that this holds true even whenthe size of the inset picture changes. Furthermore, is also notnecessary to provide completely separate memory areas for reading andwriting or to adapt the organization of the memory to the inset-picturesize.

The subclaims contain advantageous developments of the invention.

Accordingly, for example, the write and read addresses are continuouslyincremented from a first memory address up to a last memory address andare in each case reset to the first memory address again after the lastmemory address has been reached.

For this purpose, in the circuit arrangement according to the invention,it is preferably provided that the write controller and the readcontroller each have an address counter for incrementing the writeaddresses and read addresses, respectively.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details, features and advantages of the invention emerge fromthe following description of preferred embodiments with reference to thedrawings, in which:

FIG. 1 shows a block diagram of a circuit arrangement according to theinvention;

FIG. 2 shows a diagram of the signal profiles when writing to a picturememory;

FIG. 3 shows a diagram of the signal profiles when reading from apicture memory;

FIG. 4 shows a representation for illustrating the origination of aseam;

FIG. 5 shows a diagram of the signal profiles according to theinvention;

FIG. 6 shows a block diagram of a circuit arrangement for selecting amemory segment;

FIG. 7 shows an address pointer representation in the absence ofvertical decimation;

FIG. 8 shows a representation of memory division in the absence ofvertical decimation, and

FIG. 9 shows a representation of the memory outlay in the case ofdecimation.

DETAILED DESCRIPTION OF THE INVENTION AND PREFERRED EMBODIMENTS THEREOF

In accordance with FIG. 1, a main picture signal is fed to a circuitarrangement according to the invention via a first input A, and an insetpicture signal is fed via a second input B. The inset picture signalpasses to a decimation device 1 and also to a synchronization device 8.The picture signal of the inset picture, which is generally verticallydecimated, is buffer-stored in the form of fields in a memory 2 and,after read-out, is fed together with the main picture signal to aninsertion apparatus 3, which generates a composite picture signal.

The synchronization device 8 is connected to the decimation device 1 viaa first output and to a write controller 4 via a second output. A firstoutput of the write controller 4 is connected to the memory 2, while asecond output is connected to a segment buffer 7 and a third output isconnected to a display controller 6. The output of the segment buffer 7is connected to a read controller 5, to which a first output of thedisplay controller 6 is also fed. An output of the read controller 5 isconnected to the memory 2. Finally, a second output of the displaycontroller is fed to the insertion apparatus 3, the display controllerhaving an input C for synchronization signals of the main picture.

In order to write to the memory 2 an inset picture signal that has beendecimated by the decimation device 1, the write controller 4 generatesthe required addresses. In the simplest case, given an e.g. linearorganization of the memory 2, the write address pointer is incrementedafter each write operation. If the address pointer reaches the lastaddress, it is reset to the first address, under which the writingprocess is continued, this being done over the duration of the fieldwith interruptions in accordance with the decimation. In this case, apreceding picture (field) is inevitably overwritten.

With each beginning of a new field, the first write address is stored inthe segment buffer 7, which additionally stores the last address aswell. Furthermore, the field position and size are also stored in theform of lines and pixels per line in the segment buffer 7, the datarespectively stored last not being overwritten. The storage capacity ofthe segment buffer 7 is thus about double said data to be stored for afield, which in each case corresponds to a field in an instantaneoussegment and a current segment in the memory 2.

Each time the write pointer is incremented, the write controller 4additionally checks whether the new address has a specific offset, withrespect to the previously stored (not with respect to the currentlystored) address. This offset may also be zero in the case of a highvertical decimation factor. It serves essentially as a safety margin inorder to avoid overtaking as a result of asynchronous data acceptance orslightly deviating vertical frequencies. With this offset, an overtakesignal is generated which indicates whether the address has been reachedagain and hence this picture content has been overwritten.

The content of the segment buffer 7 is transferred to the readcontroller 5, by means of which the memory 2 is read beginning at one ofthe two addresses present after the start of reading at the insertionposition, so that the inset picture can be generated in accordance withits position and size in the main picture by the insertion apparatus 3.

On the basis of the overtake signal transmitted by the write controller4, the display controller 6 decides which of the two segments present inthe memory 2 is read out. The selection of the segment initially alwaysproceeds from the last segment. If overtaking took place before thestart of reading, then the instantaneous (current) segment is read out.A corresponding segment selection signal is transmitted to the readcontroller 5.

The insertion position is also calculated by means of the displaycontroller. The display controller 6 furthermore serves for correctingthe raster positions, this being effected by comparison between theraster position of a displayed picture and the raster position of astored picture and skipping or repetition of a line. Finally, thedisplay controller 6 can also be utilized for multi-picture insertion.

Before the algorithm is described in detail, firstly the terms used inthis context will be explained.

As was explained above, the decimated pixels of the inset picture arefirstly stored in the memory 2. For display purposes, they are read fromthe memory again in the temporal framework of the main picture. For thispurpose, the write and read addresses have to be generated by separateaddress counters.

FIG. 2 shows the corresponding signal profiles when writing to thememory 2. The instantaneous value of the write address counterrepresents a write pointer with a decimated inset pixel in the memory 2.The terms “write pointer” and “read pointer” shall be used for thisreason.

An acquisition window (vaqwin) begins with a pulse (vaqstart), and thewrite address pointer rises with each decimated pixel. Since only thevisible part of the inset picture is decimated, the write addresspointer rests whenever there is no valid line available from thevertical decimation stage. After the last line to be decimated, thepointer is reset to the start position. The picture of a ramp withshoulders results in the representation.

FIG. 3 shows the corresponding signal profiles when reading from thememory 2. The instant at which a start signal (vdisstart) must begenerated in order to begin reading from the memory 2 is calculated bymeans of the display controller 6 on the basis of the desired insertionposition of the decimated inset picture in the main picture. With eachline of the main picture, at the corresponding insertion position(hdisstart), a line of the decimated inset picture is then read from thememory and inserted into the main picture (video mux).

The picture of a ramp with shoulders once again results in therepresentation of the read address pointer. The ramps of the read andwrite address pointers differ in respect of their average gradient. Thelatter will be considered in detail below since it represents the rateat which a number of memory cells are swept over within a specificperiod of time.

The speed of the write address pointer changes with the decimationfactor, to be precise in such a way that the ramp of the write addresspointer becomes less steep as the decimation factor rises.

FIG. 4 shows the origination of a seam for the case of representation inthe frame mode. If only a field memory is present and the inset picturehas been decimated, the write pointer is generally overtaken by the readpointer. Since the sources for the inset picture (“insert_field”) andthe main picture (“parent_field”) are generally asynchronous withrespect to one another, a seam is thus produced. In FIG. 4, this isindicated by the point of intersection between the relatively slow writepointer and the read pointer that overtakes the latter, the upper-caseletters A, B designating the raster position of the inset picture andthe lower-case Greek letters , designating the raster position of themain picture.

From a temporal standpoint, the current field is read before the seam,while part of an older picture is reproduced after the seam. It shouldbe noted in this case that, as a result of the line interlacing method(interlaced mode), the raster position changes in the event ofovertaking, and this subsequently has to be corrected again.

The seam becomes very clearly visible in particular when a movingpicture is displayed, that is to say when the current picture and theolder picture contain different motion phases. If different standardsare used for the main picture and the inset picture, then rolling of theseam occurs. As a result of different frame frequencies, the combinationof the raster positions at the beginning of the representation changes anumber of times every second. It is a very complicated procedure toperform a correction in a manner dependent on this incorrect position.Moreover, the picture can be very erratic in the vertical direction andbe perceived as unpleasant.

The method according to the invention and the circuit arrangementaccording to the invention now allow, in particular in the frame mode, aseamless representation of pictures that have been decimated todifferent extents, without two separate field memories having to beavailable.

For the explanations below, it will be assumed that orthogonal memorydivision has been chosen. As a result, the memory is divided into lineswith fixed start addresses. The length of such a line in the memory isdetermined by the largest picture.

Furthermore, it will be assumed that the inset picture and the mainpicture are present in the same standard. The two picture sources can beasynchronous but, in terms of their time frame, should initially have nodeviations from one another. If the inset picture is not decimatedvertically and horizontally, one field memory suffices for generating aseamless picture-in-picture representation, since the two addresspointers cannot overtake one another on account of their identicalspeed. The case of raster position correction forms an exception. Thiscorrection is performed in the first line given a correspondingcombination of main and inset picture positions.

In this case, the read address pointer jumps by one line. In the courseof this jump, the situation where the write address pointer is overtakenmust be prevented. This is achieved in that there is space for twoadditional lines in the memory.

Furthermore, suitable control of the memory accesses is necessary. Thememory is written to with the fields in a circulating manner. As aresult, with each new field, the start address is shifted by the numberof additionally present lines in the direction of lower physicaladdresses. If the physical end address of the memory is reached in thecourse of writing, then a jump is made back to the start address.

FIG. 5 shows the corresponding signal profiles during write control (a)and read control (b). Accordingly, there is thus always somewhat morethan one inset field in the memory. In a similar manner to the case ofthe organization of two field memories, in this case, too, the writeaddress pointer determines the memory segment enable. The signals“vaqstart”, “vaqwin”, “vdisstart” and “hdisstart” again have the samemeaning as in FIGS. 2 and 3.

FIG. 6 shows a block diagram for selecting the respectively valid memorysegment. The circuit comprises an address counter 12, whose output isconnected to a first register 10 and to a first input of a comparator14, a second register 11, to whose input the output of the firstregister 10 is connected, and also a multiplexer 13, whose first input(line_adr_next) is connected to the output of the first register 10 andwhose second output is connected to the output of the second register11. This latter output is also connected to a second input of thecomparator 14, whose output is connected to a flip-flop 15.

For control of the read accesses, the start addresses of the old and newinset fields are stored in the registers 10, 11. With the beginning of anew field, the older of the two register contents is rejected, and theformerly new start address becomes the old address, while the presentcurrent address becomes the new start address. If the beginning of theolder inset field has been overwritten as a result of the memory beingwritten to in a circulating-manner, it can no longer be read.

For selection of the valid memory segment, the content of the addresscounter 12 (line_address) is continually compared with the start addressof the older of the two fields (line_address_cur) in the comparator 14.In the event of correspondence, the flip-flop 15 is set and the startaddress of the new field is then present at the output of themultiplexer 13. If a new field is begun, then the flip-flop is reset,and, as a result of the change of the register contents, the same startaddress as before is present at the output of the multiplexer 13, untilthis is also overwritten again. In this way, the memory enable points toa valid memory segment at every instant. The memory space that isadditionally present means that the read pointer cannot reach orovertake the write pointer even in the event of a jump on account of theraster position correction.

In order to satisfy the general requirements, however, this sequencemust be extended. Considerable deviations from the standard can occurparticularly in the case of video recorders which are operated with fastforward or rewind with picture reproduction. In this case, by way ofexample, it is also necessary to take account of the maintenance stateand the wear of the tape material. The memory control must be able tocompensate for the effects of a stretched tape and also synchronismfluctuations of the drive mechanism. However, a precondition in thiscase is that the sync pulse separation still operates correctly in thecase of such a signal.

The vertical frequency f_(V) and the horizontal frequency f_(H) arerelated through the number Z of lines as follows:f _(V) =f _(H) /Z  (5.1)

The line frequencies of the main picture (f_(Hp)) and of the insetpicture (f_(Hi)) are of interest for the algorithm. Their fluctuationsdirectly affect the writing and reading speed. The larger the linefrequency f_(Hi) of the inset picture, the more memory content iswritten per unit time. The smaller the line frequency f_(Hp) of the mainpicture, the fewer lines are read per unit time. The oppositecorrespondingly holds true.

If the same standard is used in both sources, then the followingrelationships hold true, where f_(H) is the desired line frequency:f _(Hi) _(—) _(max) =f _(H)(1+df _(Hi))  (5.2)f _(Hi) _(—) _(min) =f _(H)(1+df _(Hi))  (5.3)f _(Hp) _(—) _(max) =f _(H)(1+df _(Hp))  (5.4)f _(Hp) _(—) _(min) =f _(H)(1+df _(Hp))  (5.5)

It will additionally be assumed that no vertical picture decimation isperformed. Since the sources are again two sources that are asynchronouswith respect to one another, write and read pointers can adopt anydesired position with respect to one another. Equally, the pointers canovertake one another in both directions given corresponding combinationof the horizontal frequencies.

FIG. 7 shows the memory lines required for the respective write and readpointers, where Z_(acq) is the number of picture lines of a field thatare used for acquisition. The representation makes it clear how manylines must additionally be present in the memory in order to preventovertaking of the pointers in both directions.Z _(s2) =Z _(acq){(f _(Hi) _(—) _(max) −f _(Hp) _(—) _(min))/f _(Hi)_(—) _(max)}  (5.6)Z _(s1) =Z _(acq){(f _(Hp) _(—) _(max) −f _(Hi) _(—) _(min))/f _(Hp)_(—) _(max)}  (5.7)

The total amount of additionally required memory for the seamlesspicture-in-picture representation turns out to be:Z _(g) =Z _(acq) +Z _(s1) +Z _(s2)  (5.8)

During writing, the memory contains part of an old field and part of anew field. Through the position of the write address pointer, one of thetwo fields is enabled for reading. If fewer than Z_(s1) lines of the newfield have been written, the old field is enabled. Otherwise, thedistance from the beginning of the old field is less than Z_(s2) lines,with the result that the new field can be read.

FIG. 8 shows the memory division in the case where no verticaldecimation is performed.

If decimation is then effected in the horizontal and vertical direction,this influences the speed of the write address pointer. Thecorresponding relationships are shown in FIG. 9. The rise of the ramp issmaller in this figure.

It will initially be assumed that the writing speed changes to aconsiderable extent as a result of the decimation. By contrast, thefluctuations due to changes in the line frequency shall be small.

It follows from this that the read pointer can no longer be overtaken bythe write pointer on account of the speed difference. As a result, thememory outlay for seamless picture-in-picture representation can now bedetermined using the decimation factors and the fluctuation range of thepicture sources.Z _(s1) _(—) _(dec)(dec _(ver))=(Z _(acq) /dec _(ver))(1−f _(Hi) _(—)_(min) dec _(ver) /f _(Hp) _(—) _(max))  (5.9)

The additional requirement of lines decreases as the decimation factorincreases. The maximum emerges for a vertical decimation factordec_(ver) of 1 (vertically undecimated picture).

The possibility of the read pointer being overtaken by the write pointerwill now also be taken into account. For small vertical decimationfactors, the result is a further additional memory requirement of:Z _(s2) _(—) _(dec)=(Z _(acq) /dec _(ver))(1−f _(Hp) _(—) _(min) dec_(ver) /f _(Hi) _(—) _(max))  (5.10)

For a meaningful result, the expression in the right-hand brackets mustbe positive. The validity of this expression is thus limited to a rangeof:“1” less than equal to “dec _(ver)” less than equal to “(f _(Hi) _(—)_(max) /f _(Hp) _(—) _(min))”  (5.11)

For values which are greater than the right-hand limit, Z_(s2) _(—)_(dec) shall be set to zero.

Taking account of the validity ranges, the additional memory requirementresults from the sum of Z_(s1) _(—) _(dec) and Z_(s2) _(—) _(dec). Theexaminations made at the beginning are a special case for a verticaldecimation factor of 1.

The total memory requirement consequently turns out to be:Z _(g)(dec _(ver))=Z _(acq) /dec _(ver) +Z _(s1) _(—) _(dec) +Z _(s2)_(—) _(dec) for “1” less than equal to “dec _(ver)” less than equal to“f _(Hi) _(—) _(max) /f _(Hp) _(—) _(min))”  (5.12)thus resulting in the following:Z _(g)(dec _(ver))=Z _(acq) /dec _(ver)(3−f _(Hp) _(—) _(min) dec _(ver)/f _(Hi) _(—) _(max) −f _(Hi) _(—) _(min)/(dec _(ver) f _(Hp) _(—)_(max))  (5.14)

Otherwise, the following holds true:Z _(g)(dec _(ver))=Z _(acq) /dec _(ver) +Z _(s1) _(—) _(dec) for “dec_(ver)” greater than “(f _(Hi) _(—) _(max) /f _(Hp) _(—)_(min))”  (5.13)

The following results from this:Z _(g)(dec _(ver))=Z _(acq) /dec _(ver)(2−f _(Hi) _(—) _(min)/(dec_(ver) f _(Hp) _(—) _(max))  (5.15)

The total number of memory cells required has its maximum for a verticaldecimation factor of 1. As the decimation factor increases, the memorycell requirement greatly decreases.

It shall also supplementarily be pointed out that when the memory 2 isextended to three segments, the method according to the invention canalso be employed with a frame frequency of 100 Hz in the AABB raster.

1. A method for picture-in-picture insertion, in which a sequence ofdecimated inset pictures is written to a memory device and is read outfor insertion into a sequence of main pictures, the method comprisingthe steps of: writing inset pictures to the memory device in acirculating manner as fields under continuously incremented writeaddresses, the inset pictures being written to corresponding memorysegments beginning at corresponding writing start addresses, storing thewriting start address of each written-in field, each time the writeaddress is incremented, by comparison of the respective instantaneouswrite address with a previously stored writing start address, anovertake signal, which indicates whether the respective writing startaddress is reached again and the memory segment corresponding to therespective writing start address is overwritten, by evaluation of theovertake signal, selecting for read out the memory segment correspondingto the last writing start address stored or the penultimate writingstart address stored, and reading out the selected memory segment forinsertion into the respective main picture with continuously incrementedread addresses.
 2. The method of claim 1 further comprising continuouslyincrementing the write and read addresses from a first memory address upto a last memory address and are in each case resetting the write andread addresses to the first memory address again after the last memoryaddress has been reached.
 3. The method of claim 1 further comprising,in order to insert an inset picture into a main picture, in each casestoring the picture position and size are in the form of a number oflines and also pixels per line in a segment buffer for two insetpictures.
 4. The method of claim 1 further comprising effecting theraster correction by comparison between the raster position of a pictureto be displayed and the raster position of a stored picture and also byskipping or repeating a line.
 5. The method of claim 1 furthercomprising, each time the write address is incremented, comparing theinstantaneous write address with the penultimate writing start addressstored wherein, in the event of correspondence, the last writing startaddress stored is used as reading start address for reading thecorresponding memory segment, whereas otherwise the penultimate writingstart address is used as reading start address for reading thecorresponding memory segment.
 6. A circuit arrangement for inserting asequence of decimated inset pictures into a sequence of main pictures,comprising: a write controller for writing the inset pictures as fieldsunder continuously incremented write addresses to corresponding memorysegments of a memory device beginning at corresponding writing startaddresses, having a segment buffer for storing the writing start addressof each field written to the memory device, in which case an overtakesignal can be generated by the write controller each time the writeaddress is incremented, by comparing the respective instantaneous writeaddress with a previously stored writing start address, which overtakesignal indicates whether the respective writing start address is reachedagain and the memory segment of the memory device which corresponds tothe respective writing start address is overwritten, having a displaycontroller to which the overtake signal is fed, in which case thedisplay controller can select, by evaluating the overtake signal, thememory segment corresponding to the last writing start address stored orthe penultimate writing start address stored, for read-out by a readcontroller, connected to the segment buffer, with the aid ofcontinuously incremented read addresses and for insertion into therespective main picture.
 7. The circuit arrangement of claim 6 whereinthe write controller and the read controller each have an addresscounter for incrementing the write addresses and read addresses,respectively.
 8. The circuit arrangement of claim 6 wherein, by means ofthe display controller, an insertion position of an inset picture iscalculated and a corresponding insertion signal can be fed to aninsertion apparatus.
 9. The circuit arrangement of claim 6 wherein, bymeans of the display controller, raster correction can be carried out bycomparison between the raster position of a picture to be displayed andthe raster position of a stored picture and also by skipping orrepeating a line.
 10. A circuit arrangement of claim 6 wherein provisionis made of a comparator for comparing the instantaneous write addressprovided by an address counter with the penultimate writing startaddress stored, the output of the comparator being connected to aflip-slop for driving a multiplexer, and in that the penultimate writingstart address stored is present at a first input of the multiplexer andthe last writing start address stored is present at a second input ofthe multiplexer, with the result that, in the event of correspondencebetween the instantaneous write address of the address counter and thepenultimate writing start address stored, the multiplexer outputs thelast writing start address stored as reading start address, whereasotherwise the multiplexer outputs the penultimate writing start addressstored as reading start address.